Deep learning in the marking of medical student short answer question examinations: Student perceptions and pilot accuracy assessment

Introduction: Machine learning has previously been applied to text analysis. There is limited data regarding the acceptability or accuracy of such applications in medical education. This project examined student opinion computer-based marking and evaluated deep (DL), a subtype machine learning, scoring short answer questions (SAQs). Methods: Fourth- fifth-year students undertook an anonymised online examination. Prior examination, completed survey gauging their on marking. Questions were marked by humans, then DL analysis was conducted using convolutional neural networks. In analysis, following preprocessing, split into training dataset (on which models developed 10-fold cross-validation) test performance conducted). Results: One hundred eighty-one examination (participation rate 59.0%). While expressed concern marking, majority agreed that computer would be more objective than human (67.0%) reported they not object (55.5%). Regarding automated SAQs, for 1-mark questions, there consistently high classification accuracies (mean 0.98). For complex 2-mark 3-mark multiclass required, lower 0.65 0.59, respectively). Conclusions: Medical may supportive due its objectivity. potential provide accurate written however further research examinations required.